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The National Science Foundation is an important part of the Teacher Enhancement landscape. TEECH can serve to 
increase communication between current and future Principle Investigators and NSF program officers. NSF can help 
by providing information on funding directions, and on how shifting trends effects how proposals are evaluated. 
Mike Haney (Program Director, Elementary, Secondary and Informal Education) has started this process off by 
providing us with a current lecture on proposal development entitled 'The Good, the Bad and the Disappointed: A 
Review of NSF Funding of Computer Science Education for Secondary Schools'. Although this paper specifically 
uses computer science teacher enhancement efforts in its examples, the criteria addressed in the paper are pertinent 
in many domains of teacher enhancement. 



The Good, the Bad & the Disappointed: A Review of NSF Funding of 
Computer Science Education for Secondary Schools 
by 

Michael R. Haney, Program Director 
Elementary, Secondary and Informal Education 
National Science Foundation 



Background... 

NSF funds Basic Research and Education ... 

Within Education, there are a number of divisions. The ones that fund computer science are: 

• Division of Undergraduate Education, which funds projects for undergraduate education, which includes 
preservice teachers. (www.ehr.nsf.gov/EHR/DUE/start.htm) 

• Elementary, Secondary and Informal Education, which funds inservice education and K-12 course 
development in three programs: ( www.ehr.nsf.gov/EHR/ESIE/index.html ) 

o Teacher Enhancement 

o Instructional Materials Development (materials for student use) 
o Informal Science Education (museums, community, scientific literacy) 

Most programs at NSF require prelims. These are used as sounding boards for your ideas. They are read by several 
program officers who will return written comments and suggestions to you. For ESIE, prelims are mandatory, and a 
full proposal cannot be submitted unless a prelim was reviewed first (except for resubmissions when the guidelines 
have not changed). 

Full proposals are reviewed by a peer review panel, usually between 4 and 9 outside experts who advise NSF on 
funding. Because of this process, it is critical that the proposal you write be self-explanatory and convincing to 
people with a variety of expertise: content discipline, teaching methodologies, evaluation, school change, 
curriculum implementation, etc. 



Teacher Enhancement awards about $100M per year in teacher inservice projects, including planning, conference 
and other grants. Often a large portion of these funds are for special initiatives. In the five years I have been at NSF 
there have been funds earmarked for: 

• Private Sector Partnerships, 

• Local Systemic Change, Elementary Math & Science, 

• Local Systemic Change in Secondary Mathematics, 

• Advanced Technological Education, 

• Local Systemic Change, Secondary Science (will be coming soon...) 

The Bad You are shooting at a moving target. The expectations of proposals are changing. Many proposals funded 
'T _ three years ago would not be acceptable now. Below is a history of a few computer proposals and the reviewer 
ratings they received. Those in green were funded. 



The Good 
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The History-10 CS (?) proposals with their panel ratings. Funded are in 
green. 

description 


1992 


1993 


1994 

1st 


1994 

2nd 


1995 


1996 


18 graduate credits in mathematics and CS for a 29 month project 


EVG 

F 

2.5 






2 yr $300K upgrade cs and near-cs teachers to IEEE & ACM standards 


EEV 

GP 

2.6 


VVG 

G 

2.5 

wd 






42 mo $600K for math/cs teachers learn computational science 




VGG 

G 

2.25 


EVV 

GGF 

2.5 




22 mo $270K adding lab component to AP 


GGF 

F 

1.5 






2 yr $160K upgrade teacher content in CS 


PPPP 

P 

0 




3 weeks $47K "analytical and computational methods" 


GGG 

F 

1.3 






3 yr $1.6M computer science & integrated applications 






GGG 

FFP 

1.3 


GGG 

GFF 

vvv 

G 

2.1 






3 yr $600K teachers statewide to be local technology experts-MS 




VVG 

FF 

2 




EVV 

GG 

G 

2.7 




1 yr $100K AP teachers learn C++ 










GFF 

1.3 


3 yr $200K rural teachers to be technology experts 






GFF 

1.3 


The number of Computer Science conferences that were funded 




1 


1 


1 


1 


2 



...some of the disappointments. ..or general failings of proposals 

• Far too much is promised... 

• Projects must involve teachers at all stages of the project. 

• The project staff members are not qualified in all aspects of the project (content, pedagogy, materials, etc.). 

• A broad committed (collaboration) is needed to make this project work. 

• Computer Science is used to fragment in the curriculum (ex., another language course) rather than consolidate 
(ex., computational science). 

• The cost per student not justifiable. It should be less than $4K. 

• The project is too narrow or elitist, with little appeal to most students. 

• The goals, objectives, strategy, outcomes and evaluation plan are weak so little will be learned from the 
project. 

• NSF support would be used to expand the university course catalog. 

• PI motivation is doubtful (self serving) and not linked to the community's view of what is needed. 

• Too few details are provided, especially about workshops, what is taught and how it is taught. 

• Tuition is charged even though the project includes salaries and indirect costs. 

• Finally, it should be noted that projects that simply add a new language are an unlikely area for support. 

The following as samples from reviews of various projects. 
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1. oh, so much to do in so little time 

"Far too much is promised (covered) in far too short a time. The proposal also had many problems. It was very 
ambitious and too broad in scope. It tried to do too much for rural schools through individual teachers. The topics in 
the first year include object oriented design, GUI, libraries, e-mail, HTML, spreadsheets and others. The final year 
included events, JAVA, math, machine simulator, BNF, state diagram, Mathematica. Through all this, the 
participants have long- and short-term projects in the summer plus the additional burden of their new roles in 
schools with which to deal." 



"Another concern of the panel was that the proposed project includes a lot of material to cover in four weeks. 
Although these are very bright and motivated teachers [...], they have not encountered the computer concepts of 
objects, classes, inheritance, virtual functions, dynamic binding, and so forth. If the project is primarily for teacher 
enhancement, then the Pis need to show how this is feasible to accomplish in the summer." 

"A detailed syllabus would have helped this proposal substantiate the claim that so much could be covered in four 
weeks." 

"The project had many deficiencies in the design. For example, the teachers would have one week to learn C. This 
is unreasonable. Moreover, there was no justification for learning C provided." 

"Although the Pi's have experience with intense summer programs like this, the number of topics and speed of the 
courses undermine some of the project's strengths such as the hands-on learning. It appears teachers would have 
little time to experiment and discover when the pace is so quick. The project does not include any additional time 
compared to the prior project but does include many more topics." 

"The syllabus is overwhelming. In the first summer (5 weeks) they will study Prolog, Computer Architecture, Lisp, 
and some C++. In the second 5 weeks one year later, they will finish C++ and cover operating systems, languages 
and compilers, basic data structures, graphics, artificial intelligence, and data bases. The teachers do work on these 
in the intervening academic year independently with support through e-mail." 

2. you must involve teachers 

Teacher enhancement must involve teachers at all stages and in all phases of the project. The following are from 
reviews of projects that failed to do so. 

"The PI should have made a much stronger case for the need for the workshops, especially considering the 
attendance at the previous workshop. The model for this workshop is a very old one that NSF no longer supports. 
This is a one-shot teacher institute without any form of follow-up. Teachers were not involved in the planning of the 
new workshop and some of their important suggestions from the survey (such as more applications) seems to have 
been ignored. In fact, much of the proposal was a copy of the previously funded proposal." 

"Teachers are the clients of this proposal and not involved in the planning or administration of the project... " 

3. staff qualified in narrow areas... 

The project staff members are well qualified in their discipline(s) but not in methods and/or knowledge of schools. 
Examples follow. 

"The PI did not seem to be aware of the diverse needs of the individual teachers. He did not seem to know how 
teachers could integrate what they learn into the curriculum." 

"Many of the topics included in the course would not be of value to the typical high school teacher. Their relevance 
needs to be explained." 

"The project seems to emphasize tools exorbitantly and almost to the exclusion of pedagogy. It may be that some of 
the methodologies do transfer with the teachers to their classrooms, but little time is given to how [this would 
happen] and much time is given to the what [would happen]." 

4. partnerships... 



Who is committed to making this project work? Examples. 
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"...Moreover, the project responds to what is only an implied need without data, letters of support, or a needs 
assessment." 

"Small problems were in the limited cost-sharing, the need for specific data, and the lack of pre-college teaching 
experience. More serious were the lack of letters of support, an advisory committee, and the incomplete evaluation 
plan." 

"The evaluation was based on student performance, it depicted XXX as a partner although there was no such letter 
of support, and would compare the treated teachers' students [test] results with the national population." 

"The telecommunications component assumes that schools have the technology needed to support the teachers in the 
project. The workstation requirements perpetuate the model of one connected computer in a classroom, and the Pis 
have failed to envision anything that goes beyond the single connected computer in the school." 

"This would have been a stronger proposal if it included networking commitments and support across the 
disciplines. At a minimum, the school should provide release time, real access to computers, and administrative 
support. Other commitments that would have been convincing would include computers for science and 
mathematics, scheduled school-wide working groups, and a LAN." 

"Access to Internet is promised only during the project. Moreover, it is for lead teachers only. Who will provide 
(and what commitments have been made for) Internet access after the project? How do the peers whom these 
teachers are teaching get access?" 

"After the project is over, many of the goals will have been met. However, it is assumed that once a critical mass is 
in place good things will continue to happen among the teachers and between teachers and the research community. 
There are no commitments to sustaining the impact?" 

"The commitments made by the participant and the school district do not seem to include equipment of any sort." 
"The only letter of support is from the grants office at the university." 

5....just another fragment 

"Computer Science is one additional fragment in an already too fragmented curriculum. ..the project is disconnected 
from mathematics, science or other parts of the student's life as the following show. 

" In general, we believe (and TIMSS shows) that the American school experience is a mile wide (covers much) and 
an inch deep (achieved little deep understanding of anything). So why should computer science make this a 
mile-and-a-half wide?" 

"The panel wrote that XXX has produced good quality products in the past. If the collaborative could provide a 
model for math and science integration then the project would be of value, but the panel was not convinced that this 
project does that. The collaboration needs to include other people who can integrate the standards and their 
applications through technology and who are in tune with math and science teacher needs." 

"The most fatal flaw of the proposal is that it needs a stronger, clearer and more constructed tie to mathematics and 
science. The assumption in the proposal was that math and science teachers will be included; however, they are 
mostly taught computer science. To quote the panelists, 'The purpose of the project is to help science and math 
teachers integrate computers in the high school curriculum, but the proposal shows no evidence of how this will be 
done. The main focus of the proposal is missed by the activities. Only one course has topics relating to math and 
science curriculum.'" 



"There were several major concerns raised about the appropriateness of more fragmentation of the high school 
curriculum. Also troublesome to the panel was the ability of these Pis to address what is a complex issue, the role of 
computer literacy in schools. Another practical concern of the panel was fitting one more course into the curriculum. 
This is not a programming course, it is more history of computing, problem solving, algorithm development, and a 
look at careers. Why were these not tied into the math or the science goals. The basic premise seemed to be that 
computer science is an alternative to high school science. The panel was not convinced." 



6. cost per student 



The cost per student out of proportion with the benefit, or with the projects against which it competes for funding. 



0 
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"The proposal raised additional issues such as: Where is the math and science curriculum in this project? How is the 
curriculum developed? The cost per teacher would have been $9.18K, although approximately 720 additional 
teachers would be reached indirectly depending on the [the project]." 

7. narrowly focused or elitist... 

The project is narrowly focused or elitist. In general, AP has been considered too narrow for IMD support. 

"This appears to be aimed at building a foundation to selling a book to be published. Many text books now come 
with XXX." 

"The teams, which are a strength of the project, essentially do not exist other than in the summer. There are no 
details about how any interaction could be sustained and no plans to offer activities, networking, workshops, etc., 
during the school year." 

8. goals, implementation, outcomes 

Connect project goals, objectives, strategy, & outcomes. You need to evaluate how well these are accomplished 

"The assessment was felt to be extremely test driven with pre- and post-tests as well as projects." 

"Evaluation and long term goals were very sketchy. The evaluation plan is that the school would provide a letter." 

"The evaluation plan is quite vague. For example, in the second part of the plan (the outside evaluator) has complete 
access to all the records and materials. There is no indication what he is trying to assess. In another phase, the 
participants are invited to "comment on the impact that the project has had thus far on the teaching of mathematics 
and computer science." Again, there is no indication what goal or objective is being assessed." 

9. a cheap way to expand the course catalog 

NSF is being asked to pay for new or token additions to the university course catalog and little else is accomplished. 

"The concept of creating a knowledgeable cohort of teachers in the state on networking is excellent and this might 
be a national model, but this project includes just one Internet course and little else new. Consequently, this project 
looks much like an effort to raise enrollment and get masters degree students for the university." 

"The proposal has admirable goals but ambivalence in reaching them. Masters degrees would be the focus of the 
project to produce master or leader-teachers for the communities. However, only one course would be on 
networking, one on independent work, and the rest appear to be "off-the-shelf" courses. There is little indication that 
technology would make inroads into the full degree program, that this represents a new area of concentration, and 
that there is a plan to support the teachers in their roles in the schools. Both the courses and the modes of delivery 
lacked innovation." 

10. aligning priorities 

Is the falling CS enrollment at your college or university a convincing argument for funding? Some Pis think so as 
the following show: 

"The Pis assume that high school computer science is important for recruiting into the hard sciences and engineering 
in college. Since colleges have shown little appreciation for what preparation students have when they arrive, this 
course is designed to address those concerns. This probably is not the course to solve that concern." 

"The project seeks to address a college problem, insufficient numbers of incoming computer science students, by 
training high school teachers in computer science. In doing so, they have lowered the general requirements and 
narrowed the focus of the teachers' preparation to serve the needs of college computer science. For example, in 
mathematics, the requirements have been lowered and essentials like discrete mathematics eliminated. Discrete 
mathematics is not difficult and is basic to understanding [stuff]. It also connects the computer models to the real 
world." 

other concerns... 

O 

ERIC 
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"There was no description of any involvement of the community. No mention was made of partners, teachers, etc. 
The panelists felt that there was no way this project could be considered systemic." 

"Other concerns were that no recruitment or selection was described .." 

"No mention was made of equity and no mention of recruiting females." 

"...too often uses terms like encourage or might be." 

"The project is not a significant change from the last project." 

"The minority recruitment plan is multifaceted but does not include any direct contact with teachers or schools. 
Everything is done through mail ..." 

"Too much of the follow-up is not described. The PI visits the sites and encourages interaction among participants. 
How? To do what?" 

"No sample lesson plans were included in the proposal." 

"No curriculum for the workshop was presented, only the topics and the high level goals. For example, in the first 
year it included 8 topics but the Pis gave no insight into how this would be carried out." 

"Although the issues of involving and overcoming barriers for women and minorities have been addressed, it is not 
clear from where this expertise comes. Moreover, it appears that these will be topics discussed rather than concerns 
infused throughout the training." 

other issues 

languages are a tough sell... 

Languages do not sell easily. Other, more transferable techniques, might.. .such as problem solving, modeling, 
computational science, etc. 

C++ does lead to jobs, so why not consider ATE? Pascal had a root in structured (procedural) problem solving, C++ 
is an attempt to meet more practical computer science and employer needs. There is a conundrum here that has not 
been resolved. 

Things to remember... 

Teachers should be involved in every phase of the project, from planning to implementation to sustaining the 
project's impact. 

Have a minimum of 100 contact hours, substantial follow-up, typically supported by 3-6 graduate credits or CEUs. 
The efficacy of the courses is most easily established by graduate credit, but other methods may be OK also. 

The evaluation must be deep and thorough. The impact should be on teaching practice and students. 

Prove the project is self-sustaining. 

Because of the use of prelims, panels generally see very good proposals so the competition is keen. 

Ask yourself why Federal money should be spent supporting your need. The panel will ask that question. 

Submitting a proposal is a humbling process , and the computer education community is not accustomed to this 
process. However, there are consistencies that make navigating the waters a bit easier. The experience over the 
past five years has produced some understandings about what could be fundable. Paying attention to what these 
panels have told us and the newest guidelines can improve your chances considerably. Submitting a prelim to NSF 
gives you an opportunity and a contact for extended conversations about the project you are proposing. It should 
improve your chances of not making a fatal proposal mistake. 



From a talk delivered at the George Washington University workshop for future Principal Investigators in San Jose, 
CA, March 1, 1997 by 
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